HYBRID TRACE FORMULA FOR A NON-UNIFORM 
IRREDUCIBLE LATTICE IN PSL 2 (M) n 



DUBI KELMER 

(3JT)[ Abstract. In [KelOj we used a version of the trace formula, on 

■ the space of hybrid Maass-modular forms of an irreducible uni- 

form lattice T C PSL2(K) n , to study the distribution of holonomy 
classes of closed geodesies on the quotient r\PSL2(K) n . In this pa- 
per we derive the analogous formula for a non-uniform lattice and 
deduce the same equidistribution results for the holonomy classes. 
For Hilbert modular groups, there is a nice interpretation of these 
results in terms of class numbers and fundamental units of qua- 
dratic forms. 



Introduction 

Let T denote a lattice in PSL 2 (IR). For any hyperbolic 7 G T (i.e., 
with |tr( 7 )| > 2) let p 7 > 1 such that 7 ~ x/ ^-\. The Selberg 

Zeta function of Y is then defined by the Euler product 

00 

z{sj)= n H(i-p~ s+k ), 

{i}er' h fc=o 

where the first product is over all primitive (i.e., not a power of an- 
other element) hyperbolic conjugacy classes. This product converges 
for 9ft(s) > 1, and using the Selberg trace formula Selberg |Se56] showed 
that Z(s, T) has an analytic continuation to C and satisfies a functional 
equation relating Z(s, T) and Z(l — s, V). Furthermore, its zeros in the 
critical strip < 3?(s) < 1 are located on the line {| + iM} U (0, 1] and 
are related to the spectrum of the hyperbolic Laplacian on L 2 (r\H). 
Here H denotes the upper half plane endowed with the hyperbolic met- 
ric and T acts by isometries via linear fractional transformations. 

Now, let T C PSL 2 (M) n denote an irreducible uniform lattice. In 
his 1995 lecture |Se95] , Selberg described how to construct partial Zeta 
functions from the primitive elliptic-hyperbolic conjugacy classes in T. 
Here, an element 7 G T is called elliptic-hyperbolic if |tr( 7 j)| < 2 for 
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j = 1, ... ,n — 1 and | tr(7„)| > 2. For an elliptic-hyperbolic 7 G T, 
let p 7 > 1 correspond to 7„ as above and for j — 1, . . . , n — 1 let 7^ ~ 

with e 7 . G S* 1 . For any weight m G N 71-1 , the corresponding 



A 
I S J 



"7? 

partial Zeta function is given by 

z m (s,r)= n n ( i -< i ---^:^r fc ") (_l) "" 1 > 

{7}er; h fc„>o 

|fej|<rrtj 

where the notation T' eh indicates that we only consider primitive elliptic- 
hyperbolic classes. 

Just as in the classical setting of the Selberg Zeta function, the an- 
alytic properties of these partial Zeta functions are derived from a 
corresponding trace formula. Specifically, this is a trace formula on a 
space of hybrid Hilbert Maass-modular forms that we will now describe: 
Consider a product, HP, of n hyperbolic planes. For any m G 2P let 
L 2 (r\HP,m) denote the space of functions ip on HP which are square 
integrable on r\HP and satisfy ^("fz) = j 7 (z) m ip(z) for all 7 G T. Here 
we used the shorthand 

where j^(zj) = for 7,- = ( a c b d ). Let A Zj , mj denote the partial 

mj-Laplacian acting on the fth coordinate, 

d d s d 



j j j 



For m G N^ 1 let M m (T) denote the subspace of L 2 (r\HP,m) com- 
posed of joint eigenfunctions of A ZjtTnj , j = l,...,n — 1 with cor- 
responding eigenvalues |?7ij|(l — \m,j\). Here and below we naturally 
identify N n_1 C Z™ -1 C Z n by making the last coordinate zero. We 
similarly define the spaces M am {T) for all possible signs o G {±l} n_1 . 

For m G N n_1 and a G {±l} n_1 let {^ m) } fceN be an orthonormal 
basis for M am (T) composed of eigenfunction of A Zn with eigenvalues 

< Ao(am) < Ai(crm) < X 3 (am) .... 

Remark 0.1. Recall that for a lattice T in PSL^R), the Maass wave 
forms are the Laplace eigenfunctions in L 2 (r\HI), and that the space 
of modular forms of weight 2m can be identified with the subspace of 
L 2 (r\H, m) composed of eigenfunctions of A m with eigenvalue m(l — 
m) (via the map f(z) 1— > y m f(z)). Correspondingly, we can think of the 

functions as hybrid forms, behaving like Hilbert modular forms 
of weight m in the first n — 1 coordinates, and like Maass forms in the 
last coordinate. 
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For r C PSIj2(IR) n an irreducible uniform torsion free lattice, the 
trace formula for these hybrid forms takes the following form (see |KelO[ 
Theorem 7']). 

oo 

o-ejii}' 1 - 1 fc=i 

|m|*voi(r\e n ) 



2(2vr)" 



h(r)r tanh(7rr)o?r 



h(llog(p 7 )) 



n-l 




+ (-ir 1 E Emp.) / 2 / 2 __ 

where /z(r) is any even holomorphic function with Fourier transform h 
compactly supported, Afc(m) = ~ + r^ m , and |m|* = rij=i(2| m il — !)• 

Remark 0.2. The no torsion assumption is only used to simplify the 
exposition, when T has torsion there is a similar formula with an ad- 
ditional term corresponding to the elliptic classes. The assumption on 
the compact support of h can also be relaxed. The sum over all pos- 
sible signs a G {il}"" 1 is also not necessary, and there is a similar 
formula without it. We chose to present the formula in this form to 
emphasize the resemblance to the logarithmic derivative of the partial 
Zeta function 

7' fc n 00 „(i/2-*)' 

§#R = (-ir 1 EE 1 °s(^ ' 

From this trace formula, using the standard arguments, we get that 
Z m (s, T) has an analytic continuation and functional equation relating 
s, 1 — s. The zeros of Z m (s,T) are the trivial zeros located on the 
negative integers, and the spectral zeros located at s G C such that 
s(l — s) = Afc(crm) with order equal to cr\s(l — s) = \k(<rm)}. In 
the special case where all \m,j\ = 1 and n is odd (respectively even) 
there is another zero (respectively pole) at s = 1 of order 2 rt_1 . 

Remark 0.3. The partial Zeta function is actually the square of an 
analytic function defined by taking the product over just one out of 
each pair of inverse classes (note that the contribution of {7} and {7 -1 } 
to the Zeta function is the same). Moreover, under certain assumption 
on the lattice (specifically, if it is derived from a maximal order in a 
quaternion algebra), it follows from |Kel0t Corollary 6.1] that Z m (s, T) 
is actually a 2 n_1 power of an analytic function, which explains the 
order of the zero/pole at s = 1. 
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In addition to the application to the partial Zeta functions, the hy- 
brid trace formula was used in |KS09j in order to establish the existence 
of a strong spectral gap for r\PSL 2 (K) n . In particular, the bounds on 
the strong spectral gap imply that Xo(m) > c(T) > for all m G Z n_1 ; 
this can be interpreted as a uniform zero free region for all the partial 
zeta functions Z m (s, T), m G N n_1 . The hybrid trace formula was also 
used in |KelO] to compute the asymptotics of the number of elliptic- 
hyperbolic classes (with p 7 bounded), and to study the distribution of 
the corresponding elliptic partes e 7j in S 1 ; under the correspondence 
between closed geodesies and conjugacy classes these elliptic parts give 
the holonomy class of the corresponding closed geodesies. 

The most famous examples of irreducible lattices in PSL2(M) n are the 
Hilbert modular groups, IV = PSL 2 (Cf), where Of denotes the ring 
of integers of a totally real number field of degree [F : Q] = n. These 
lattices are non- uniform (that is, the quotient IV\IHP is not compact), 
and hence |KelOt Theorem 7'] does not apply directly. Nevertheless, 
it was shown in [KelOj that when n > 2 there is a companion uniform 
lattice IV, such that the geometric side of the hybrid trace formulas for 
Tp and IV are identical. Using this observation, the results of |KelOj 
on counting elliptic-hyperbolic classes and distribution of holonomy 
angles, as well as the properties of the partial Zeta functions still hold 
for IV as long as [F : Q] > 2. 

If we replace Tp with a finite index subgroup it is less clear how to 
find such a companion uniform lattice, moreover, for quadratic fields 
such a companion uniform lattice does not exist even for the full group. 
In order to deal with these cases, we need to use the hybrid trace 
formula for an irreducible non-uniform lattice T C PSL^IR)™. 

In this case, there is an orthogonal decomposition 

L 2 (r\H n , m) = L 2 c {T\M n , m) © ^(r\H n , m), 

into the continuous spectrum (exhausted by the Eisenstein series) and 
the discrete spectrum (spanned by the cusp forms and residual forms). 
We then define M m (T) C L^{T\M n , m) as above (using only the discrete 
part of the spectrum). In order to derive the hybrid trace formula 
here, we need to account for the continuous spectrum, the parabolic 
elements, and the hyperbolic elements having a parabolic fixed point, 
and their contributions to the trace formula. We will show that when 
n > 2 all these new contributions cancel out perfectly and we recover 
exactly the same formula as in the uniform case. (This cancelation is 
not very surprising in view of the results for Hilbert modular groups). 
The case of n = 2 is more interesting as some of the new contributions 
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do not cancel out and we get terms that did not appear in the uniform 
case. Specifically, in this case we have 

Theorem 1. Let T C PSL 2 (K) 2 denote an irreducible torsion free lat- 
tice with k inequivalent cusps. For any even holomorphic function h(r) 
with Fourier transform h compactly supported, and any m G N 

oo „ 

V h(r k , m ) - <WMf) = (2m "(y " 2) / Kr)r tanh(7rr)rfr 
fc=i 

-\ E E'ogW^^( E 4* 

7 er^ i=i v |fc|<m 

K / OO 

- ^ fl« ( fr(0) + 2h(2lR®) eiq>(-2lR®(m - |)) 

where RS 1 ' > denotes the regulator of the i 'th cusp. 

Remark 0.4. The assumption of no torsion is only used to simplify the 
exposition and the assumption on the compact support of h can be 
relaxed; see Theorem [3] below for the general formula. Note that the 
spectrum of A 22 on M m (T) and M_ m (r) is the same so there is no need 
to sum over these two sign changes. 

The analytic properties of the partial Zeta functions Z m (s,T) for a 
non- uniform irreducible lattice T C PSL2(lR) n follow from the trace 
formula as in the uniform case. The only difference is when n = 2, in 
which case Z m (s, T) has additional double zeros at 1 — m + j^i, k G Z 
corresponding to the additional terms coming from the cusps. 

Remark 0.5. While working on this paper we learned that Gon |Gol2] 
also derived this formula for the Hilbert modular group PSL 2 ((9p) with 
F a quadratic field of class number one. He then used it to study the 
corresponding partial Zeta functions and counting function of elliptic- 
hyperbolic elements in this setting. We note that his definition of the 
Zeta functions is slightly different from ours. 

We conclude the introduction by presenting a few consequences re- 
garding the counting function of elliptic-hyperbolic classes and the dis- 
tribution of their elliptic part for a non-uniform lattice. The following 
is an improvement of (KelP} Corollary 6.2] 
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Theorem 2. ForT C PSL 2 (IR) n a non-uniform irreducible lattice and 
a smooth function, f , on (S l ) n ~~ l 

/(*r) = 2"" 1 Li(T)^- 1 (/) + O f (T^), 

p 7 <T 

where ji is the measure on S 1 = IR/27rZ given by dfx(9) = sin 2 (|)^, 
andU(T) = fij^. 

Remark 0.6. For T = PSL^CV) with [F : Q] > 2 this was proved in 
[KelOl Corollary 6.2] under the additional assumption that / is even 
in two of its coordinates. Our new result removes this assumption 
and is also valid when F is a quadratic extension and V an arbitrary 
irreducible lattice. 

Paying a small price in the exponent of the error term, we can also 
allow for a sharp cutoff. 

Corollary 1. There is 5 = 5k > such that for any k arcs I\, . . . , Iy. 
in S l 

#{{7} e r ch |p 7 < T} 

where the implied constant does not depend on the arcs. 

Remark 0.7. Assuming the Selberg-Ramanujan conjecture we can take 
5k = 2 (k+2) ■ Unconditionally using the known bounds for the spectral 
gap, due in this setting to Blomer and Brumley jBBllj . we can take 
= e>^2 = I an d 5k = 64 (fc 5 | _ 1 ) for A; > 2. Since the error term in this 
result does not depend on the choice of arcs Ij, we can also let the arcs 
shrink with T, at any rate slower than the error term of 0(T~ Sk ). 

For the Hilbert modular group, IV, with F a totally real number 
field, there is a nice interpretation of these results in terms of class 
numbers and fundamental units of quadratic forms. Fix a field em- 
bedding t : F ^ W 1 compatible with the identification of Tp C 
PSL2(lR) n . Given a binary quadratic form q(x,y) = ax 2 + bxy + cy 2 
with a,b,c G Of, its discriminant is D = b 2 — Aac, its divisor is the 
ideal (a, b, c), its splitting field is F(y/D) and its primitive discriminant 
is the ideal d = , . We say that two forms q,q' are equivalent if 
q'(x,y) = tq((x,y)^) for some t G F* and 7 G IV. The splitting field 
and the primitive discriminant are invariant under this equivalence re- 
lation and we denote by h(D, d) the number of equivalence classes with 
splitting field F(y/D) and primitive discriminant d. To any D G Op 
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with Lj(D) < for j < n and L n (D) > and an ideal d C Of, 
there is an associated fundamental unit e D d G @f(Vd) satisfying that 

\ L j(tD,d)\ 2 = 1 for j < n and i n (tD,d) > 1 (this is the fundamental 
solution to a certain Pellian equation; see section for more details). 
We then have 

Corollary 2. For any smooth function f on (S l ) n that is even in each 
coordinate (i.e, invariant under €j h- >■ ej 1 ) 

h{D, d)f(n(e D j), in-i{e D4 )) = 2 n - 2 f,(f)Li(T 2 )+O f (T 3 / 2 ), 

(D,d) 
<-n{tD,d)< T 

where the sum is over discriminants D G O f modulo F 2 that are neg- 
ative in the first n — 1 places. 

Remark 0.8. When [F : Q] > 2 this result already follows from |Kel0t 
Corollary 6.3]. 



1. Notation and preliminaries 

We use the notation A = 0(B) or A < B to indicate that A < cB for 
some constant c, and we use subscripts (e.g., A = O t (B) or A < e B) to 
indicate that the constant c = c(e) may depend on the parameter. We 
also use the notation A(T) = o(B(T)) to indicate that A(T)/B(T) — > 
as T -+ oo, and A(T) ~ B(T) if A(T)/B(T) 1. 

1.1. Coordinates. Let C7 = PSL 2 (R). For any z,t G R and G 

[— 7r, 7r] let 

A x\ fe 1 ' 2 \ , / cos(f) sin(f)\ 

"■=(,0 lJ' Gi= ^0 e-/ 2 J'^ = ^-sin(|) cos(l)J- 

Any g G G has a unique decomposition g = n x a t k e . In these coordi- 
nates the Haar measure of G is given by dg = e^dxdtdQ. 

There is a natural action of G on the upper half plane H = {z = 
x + iy\y > 0} by linear fractional transformations (g = ("a) sends z to 
gf). The stabilizer of i G H is if = SO(2) and we can identify H = 
G/K. For this identification it is convenient to use the notation p 2 = 

n x a ln[y) = for z = x + i y e H - For an y G PSL(2,M) we 

also have a decomposition g = p z kg, and in these coordinates we have 
dg(z, 9) = dzdO, where dz = ^P- is the hyperbolic area on HI. 
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1.2. Invariant operators. The ring of left G invariant differential 
operators is generated by the Casimir operator, Q, given in the z, 9 
coordinates by (see |La85| Chapter X §2]) 

d 2 d 2 d d 

For any m G Z let \m denote the characters of K = SO (2) given by 
Xmikg) = e im6 . We say that a function ip on PSL 2 (M) has right K-type 
m (or wight m), if ip{gk) = ip(g)x m {k). The ladder operators 

(1.2) ^ ±2iye± <^ Ti | )T2ie± <«iL 

commute with Q and act as raising and lowering operators between the 
different weights. That is, if ip is of weight m, then C^ip is of weight 
mil. Moreover, if ip an eigenfunction of Q of weight m and eigenvalue 
A then (by [La85l Chapter VI §4,5]) 

C^Fij) = 4(±m(l Tm)- 

For m G Z, consider the function ip S)7n on PSL 2 (R) defined by 

(1.3) ^s, m {n x a t k e ) = e st e im9 . 

This function is of weight m and is an eigenfunction of Q with eigen- 
value s(l — s). We also note for future reference that it transforms 
under the ladder operators via 

(1.4) C^Vs,™ = 2(s ± m)ip S)m 

Remark 1.1. We can identify a function ip of weight m, with a corre- 
sponding function on EI given by ip(z) = ip(p z )- The condition that 
"0(7fiO = 4>{9) f° r some 7 = {% b d ) G PSL 2 (IR) is then equivalent to 
the condition that t^("fz) = ip(z)j 1 (z) m , with j y (z) = With this 

identification, the Casimir operator Q becomes the m-Laplace opera- 
tor A m = y 2 {-§^2 + JpO — 2«m?/^:, and the ladder operators become 
£^ = ±2y(i^ ± J^) ±2m. This identification is convenient for certain 
calculations as well as for comparison with some of the literature. 

1.3. Products. We now consider a product of n > 1 copies, G = 
JT^ =1 Gj', with each Gj = PSL 2 (R). Any g G G is of the form g = 
(<7i, . . . , g n ). We use coordinates on G coming from each factor as 
above (e.g., k e = (k 01 ,. . . , k 9n ) for 6 G [-7T, 7i] n , a t = (a tl , ■ ■ ■ , a tn ) for 
t G R n etc.). 

For m G Z n let Xmikg) = e m ' 8 . We say that a function f on G 
is of wight m if ip(gk) = il>(g)Xm(k). The ladder operators Cf send 
functions of wight m to functions of weight m ± ej and commute with 



HYBRID TRACE FORMULA 



Qj (and clearly also with Cf and fij for all % ^ j). Here and below, 
ej G Z n denotes the unit vector with 1 in the j'th entry. 

1.4. Conjugacy classes. Let G = YYj=i Gj be as above, and r C G an 
irreducible lattice (that is, T is a discrete subgroup with vol(r\G) < oo 
and such that the projection to any factor of G is dense). The T- 
conjugacy classes are classified into the following types: Hyperbolic 
(when all factors are hyperbolic), Elliptic (when all factors are ellip- 
tic), Parabolic (when all factors are parabolic) and mixed (when some 
factors are hyperbolic and some are elliptic). We call a hyperbolic el- 
ement regular if it does not have the same fixed point as a parabolic 
element of T, and we call it hyperbolic-parabolic if it does. 

The conjugacy classes that will eventually contribute to the hybrid 
trace formula are the elliptic classes, r e , and the elliptic-hyperbolic 
classes, r e h, which are the mixed conjugacy classes that are elliptic in 
the first n — 1 places and hyperbolic in the last. In particular, any 
elliptic-hyperbolic class 7 G T e h is conjugated in G to (a&^,kg) for 
some i 1 > and 9 y G (— 7r,7r) n_1 (correspondingly, e 7j = e %e ~'' ] l 2 and 
p 7 = e^ 7 ). Similarly, for an elliptic class we have that 7 ~ kg^ for 
$ 7 G ( — 7i, 7r) n . 

Remark 1.2. For e G S 1 the matrices and (q") are conjugated 

PSL 2 (C). However, the matrices k g and k^ 1 are not conjugated in 
PSL 2 (R) and we can use the relation e 7j . = e ldl ' ] to to define e 7j unam- 
biguously. 

2. Point pair invariants 

We recall some of the properties of point pair invariants on the hy- 
perbolic plane and their lifts to functions on PSL 2 (M). Let p be a 
smooth compactly supported function on (0, 00). For any m G Z we 
define a function on EI x EI as in [He83_, Page 386] by 

(2.1) />, w) = (-i)-p ( ';; w f ) 

\Lm{z)Lm{w) J 

For any g G PSL 2 (IR) we have f(gz,gw) = jg(z) m f(z,w)j g (w)~ m and 
we can extend / to a function / on PSL 2 (R) x PSL 2 (R) by setting 

f(Pzh,p w k 2 ) = Xm{hk2 l )f{z,w). 

We call a function obtained in this manner a wight m point pair in- 
variant. Such a point pair invariant defines an integral operator 

L f?P(g)= / f{9,9')W)dg', 

Jg 



z — w 
z — w 
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sending functions of wight m to functions of wight m and vanishing 
on functions of other .ff -types. Moreover, any fi-eigenfunction if) of 
wight m and eigenvalue A = \ + r 2 is also an eigenfunction of Lf with 
eigenvalue S m f(r), where S m (f) denotes the Selberg transform of / 
given by 

(2.2) S m f(r) = [ j{i,z)\m{z)^ lT dz 

Ju 

(see [He76l pages 364 and 392]). 

The functions h(r) = S m f(r) obtained in this way are in the Paley- 
Wiener space PW™(C) of even holomorphic functions of uniform ex- 
ponential type (that is, the Fourier transform h is even smooth and 
compactly supported). Moreover, given h G PW™(C), for any m G Z n 
there is a (unique) weight m point pair invariant satisfying S m f = h. 
Specifically, given h G PW W (C) let h(u) = ± j^h(r)e~ iru dr be its 
Fourier transform (so that h G C^°(R) and h(r) = j R h(u)e tru du), and 

define the auxiliary function Q G C£°(0, oo) by h(u) = Q(4sinh 2 (w/2)). 
We can then recover the functions p and Q from each other by the re- 
lations (see |He83t page 386]) 

(2.3) P (y) = -- [ Q\y + t 2 ) 

^ Jr 



(2.4) Q{w) = [ p(w + v 2 ) 

Jr 

2.1. Orbital integrals. Fix h G PW t0 (C) with Fourier transform h G 
C£°(R). For any m G Z, let f m denote a weight m point pair invariant 
such that S m f m = h. For 7 G G let G 1 denote the centralizer of 7 in 
G and consider the orbital integral 

(2.5) l m ( T) h)= fm{g,ig)dg. 

Jg-,\g 

Here dg denotes an (appropriately normalized) measure on G 7 \G ob- 
tained from Haar measure. Note that I m {l', h) only depends on 7's 
G-conjugacy class. We then have 



y + 4 + t 2 - t 



du. 



\/W + 4 + iu 
y/w + 4 — iu 



du. 
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Lemma 2.1. For any m G Z 

" _1 r e^h'(u) d j 
4vr JR sinh(«/2) " ' 

( 2 - 6 ) U75/*)=< 7 

where 

(2m-l)(« + ifl) 

(2.7) /i(fl,m) = i / &(u) 

4 ./TCP 



e 2 ie" 



cosh(n) — cos(#) 



du. 



In particular, Io(l; h) — 4- L /i(r)r tanh(7rr)dr and /or m ^ and 
a = sgn(m) = -S- ioe have for the trivial class 



m — 

7 m (l; fc) - J m _ CT (l; /i) = L_i__2 ^(id^ _ i)), 

Z7T 



/or 7 ~ kg elliptic 



I m {kg; h) - I m - a {k e ; h) = _ h(i(\m\ - -)), 

and for 7 ~ at hyperbolic I m (ai; h) does not depend on m. 

Proof. For the formula for I m {l] h) see [He83l page 396] when 7 = 1, 
[He83j equation 6.19 on p. 389] when 7 ~ a £ and [He83j equation 
6.30 on p. 394] when 7 ~ kg. The dependence on m for the trivial and 
elliptic classes then follow from these formulas by a direct computation. 

□ 

Two additional integral transforms that we will need are 

/"OO 

(2.8) I p m {h)= / f m {l,n x )\og{x)dx, 

Jo 

and 

(2.9) lt P (t]h)= [ f m (n x ,a t n x )\og(l + x 2 )dx. 

In particular we are interested on their dependence on m; we show 
Lemma 2.2. For m e Z with a = sgn(m) 7^ ; 

p (h) - p (h) = 2 l m l - 1 f dt - -hu 2 ^- 1 ) 

Proof. The integral (12. 8p is computed explicitly in |He76t page 406-411] 
and the dependence on m follows from this computation. Instead of 
repeating this computation here, in section 14.91 below we give another 
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proof of this identity that uses the trace formula and does not require 
the evaluation of I^(h) explicitly. □ 

It is also possible to compute (12.91) explicitly in terms of h. However, 
for our purpose the following vanishing result will suffice. 

Lemma 2.3. If supp(h) ^ then I%(t; h) = for \t\ > f . 

Proof. Using (12. ip for the point pair invariant we can write 

xa + i(3 



f m (n x ,a t n x ) = (-!)"> (a 2 {x 2 + 1)) 



xa — i(3 



with a = 2sinh(t) and = 2cosh(t). The assumption that h(u) = 
Q(A sinh 2 (-u/2)) is supported on [—a, a], implies that Q(w) is supported 
on (0,4sinh 2 (|)) C (0,a 2 ). Hence, Q'{w) = for w > a 2 and from 
(12.31) also p(y) = for y > a 2 , so that indeed 



;-l) m /^(t; h) 



p(a 2 (x 2 + 1)) 



xa + i(3 
xa — i(3 



log(l + x 2 )dx = 0. 



□ 



We finish this section with another integral identity that we will need 
(under a similar assumption on the support of h). 



Lemma 2.4. Fix a, b > 0, and let h G C, 

on [—a, a]. Then 

b 



be even and supported 



71 



h(f)cos(at) dt = e _ abh(ib) 



Proof. Substitute h(t) cos(at) = | j R {h(u + a) + h(u — a))e lut dt to get 



h(t) cos(at) 
vr Ju b 2 + t 2 



dt 



b 

2^ 

1 

2 




Aut 



(h(u + a) + ft(-u — a)) 



6 2 + t 2 



dudt 



(h(u + a) + ft.(it — a)) 



6 2 + t 2 



dt du 



Using the residue theorem we get ^ J R b f" 2 dt = e fc '"L Hence 



h(t) cos(at) 
b 2 + t 2 



dt 



(h(u + a) + h(u-a))e- bH du 
(h(u + a) + h(u - a))e~ hu du 

POO 

h(u)e- h{u+a) du+ I h(u)e~ b( - u - a) du 
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From our assumption that h is supported on [—a, a] the second term 
vanishes and the first one equals e~ ab h(ib). □ 

3. ElSENSTEIN SERIES 

We now recall the construction from |Ef87j of the Eisenstein series 
spanning L;?(r\IHP), and use the ladder operators to obtain the corre- 
sponding Eisenstein series for L^(r\H n ,m). 

3.1. Eisenstein series at oo. We first describe the Eisenstein series 
corresponding to the cusp at oo = (oo, ...,oo). Let denote the 
stabilizer of oo in T, so that a typical element of r ro is of the form 

u a\ _ ( (u\ a±\ fu n a T 

o u) ~ {{o «rV'"'' V° U ~n 

with Y[j \ u j\ — 1- 

Consider the lattice O C M™ defined by 

O = {a G R n \ n a G Too}, 

and let v = vol(M n /0). The group 

u=! [U \3a, ^ u a ^j er c 

is a free group of rank n — 1 and after fixing a set of generators 
ei,...,e n _i we can identify U with Z n_1 . Specifically, for q G Z™^ 1 
we denote by u q G U the element u q — ef 1 • • • e^~l • For j — 1, . . . , n let 
6jj denote the j-th coordinate of 6j and consider the matrix 



'h log |eii| ••• log|e 



n-1,1 



(3.1) D 



log | ex „) | ••• log|e n . 



l,n) 



We define the regulator of the cusp at oo as R = | det(D)|, and note 
that it does not depend on the choice of basis. 

Given q G Z n_1 let rj(q) = (^i(g), ... , T] n {q)) be defined by the equa- 
tion 

(s + Tcirj(q))D = (s, mq), 

where we use the notation s + 7iir](q) = (s + nir}i(q), . . . , s + niri n (q)) 
and (s, niq) = (s, niqx, . . . , iriq n -i). 

For m G Z n and q G Z™ -1 consider the function 

(3-2) <Pa,m(g, q) = II ^s+Tri^C^.mj 



14 
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where ( -p s +-Kiridq),m' is given in ( 11.3H . Note that this function is of weight 
m, it is invariant under and it is a joint eigenfunction of Qj, j = 
1, . . . , n with eigenvalues Sj(l — Sj) where Sj = s + irir)j(q). 

For q G Z n_1 , m G Z n and s G C with dt(s) > 1, the weight m 
Eisenstein series at oo is given by the following absolutely convergent 
series 

E m (g,s,q) = Vs, m (l9,q)- 

76roo\r 

For m = these are the Eisenstein series defined in |Ef87l Chapter 
II], and we can use the ladder operators (see (11.41) ) to get that for any 
meZ n 

CfE m (g,s,q) = 2[m j ± (s + 7rir)j(q))]E m±e .(g, s, q). 

3.2. Eisenstein series at other cusps. Let £i, . . . ,£« denote a com- 
plete set of inequivalent cusps of T. For each i — 1, . . . , k, let Tj C T 
denote the subgroup fixing £j. Let 7* G G send & to oo, then a typical 

element of TiT^f 1 is of the form f ^ ■ The se ^ °f M S obtained 

in this way forms a multiplicative group Z/W, and the set of a's with 
n a G TiTiT^ 1 form a lattice 0«. By fixing a set of generators for 

we define as before the matrix as well as the elements 
and r]^\q) satisfying (s + 7iir]^(q))D^ = (s,iriq) for any q G Z™ -1 . 
Let R il) = \det(D^)\ and w w = vol(M n /C (i) ). The group and 
hence also R^\ don't depend on the choice of t,;, but the lattice 
and volume do; we can (and will) choose T; so that the products 
yO)^W = vR are all the same. We can now define the Eisenstein series 
at the z'th cusp by 

E^\g,s,q)= fsU T ^9,q), 
7er\\r 

where is defined as in f 13 . 2 j) using rj^(q). 

3.3. Scattering matrix. We can write each Eisenstein series in a 
Fourier decomposition relative to the coordinates in each of the cusps. 
The constant terms in these decompositions are of particular impor- 
tance and we denote them by 

E$P(jg,8,q) = i / E$(n x T jg ,s,q)dx. 

We recall the following results from |Ef87| Capter II, Section 1] regard- 
ing the analytic continuation and functional equation of the Eisenstein 
series (for the case m — 0). 
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Proposition 3.1. The constant terms of the (spherical) Eisenstein 
series E$\g, s, q) are of the form 

(3.3) E^\g, s, q) = 5 ij¥ >%, q) + <j>^(s, q)^f_ s>0 {g, -q), 

The functions <f>^'(s,q) as well as the Eisenstein series, have a mero- 
morphic continuation to C and satisfy the functional equation 

(3.4) E (g, 1 - 8, -q) = q)E (g, s, q), 

where the scattering matrix q) is the matrix with (i,j)- coefficients 
0O,j)( S) g) ; an d E (g, s,q) is the column vector with i-th entry E^\g, s, q). 
Moreover, the scattering matrix satisfies that $(| + it, q) is unitary and 
that $(s, g)$(l - s, -q) = I. 

We can use the ladder operators Cf to get a similar result for the 
Eisenstein series for any weight. Specifically, for m G Z consider the 
rational function 

(3.5) P m {s) = Yl 



k-l + s 

k=i 



For m G Z n we consider the multivariable function P m = J| . P mj and 
define the weight m scattering matrix $ m by its entries 

(3.6) <f>Sl j \s,q) =<P^\s,q)P m (s + 



Then, starting from m = and applying £^ to (13. 3p we get 



j 



Corollary 3. The weight m Eisenstein series satisfy the functional 
equation 

(3.7) E m (g, l-s, -q) = $ m (s, q)E m (g, s, q). 
and its constant terms satisfy 

(3.8) E^\g, s, q) = 6,,^;jg. q) + <j>^\s, q)tpt., n (SI, 1 " s ' "ff)- 

3.4. Spectral decomposition. For m G Z n let L 2 (r\G,m) denote 
the subspace of L 2 (r\G) composed of functions of weight m, which can 
be naturally identified, as in remark [TTTl with L 2 (r\H n , m). For each 
m G Z n , q G Z" _1 and each cusp, we have an isometry from L 2 (0, oo) 
into L 2 (T\G,m) defined on the dense set of smooth compactly sup- 
ported functions by p >->■ J °° p(t)E$(g,l + it,q)dt. Let L 2 (r\G,m) 
denote the space spanned by the images of these isometries from all 
cusps and all q G Z n_1 . This space captures the continuous part of the 
spectrum. The discrete part L 2 d {T\G,m) is then defined as its orthog- 
onal complement and it further decomposes as the finite dimensional 
residual space spanned by the residues of Em{g,s,0) with s G [0,1], 
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and the space of cusp forms composed of functions that vanish at all 
cusps (see [Ef87j Theorm 8.4]). 

The operators Qj act on L d (T\G,m) and we can decompose it as a 
direct sum of joint eigenspaces of Q 1; . . . , O n 

L 2 d (T\G,m)= Vr(m, A), 

AeA(m) 

where A(m) C M n is the set of joint eigenvalues and Vr(m, A) the joint 
eigenspace corresponding to A = (Ai, . . . , A n ). We denote the dimension 
of each joint eigenspaces by d(m, A) = dimVr(m, A). 
Now for m G Z n_1 with all rrij ^ define the space 

M m (T) = {ifj £ L 2 d (T\G,m)\(nj + K|(l - \ mj \))iP = 0, Vj < n}, 

where we identify Z n_1 C Z n by putting a zero in the last coordinate. 

4. The Hybrid trace formula 

For m G IT 1 - 1 with all rrij ^ let M m (T) be as above. Let {V^lfc^o 
be an orthonormal basis for M m (T) composed of eigenfunctions of Q n 
with eigenvalues 

< A (m) < Ai(m) < ... 
and use the parametrization Afc(m) = \ + rl m . We also denote by 



m 



[7^(21777,^1 — 1) and define the function 



ra-1 



#m(#) II 1 _ e ieg/x(m j )e j ' 

Let ^io( r ) be an even function that is analytic in a strip |Im(r)| < | + <5 
and decay like |/7o( r )| = C((l + 3R(r)) _2 ~ <5 ) for some 5 > 0. We will 
show 

Theorem 3. For n > 2 

^^^o( r fc,ra) — ( — l) n ^m,sgn(m)^o(f ) 



k 

|m|*vol(J-r) 
(4vr) n 



ho(r)r tanh(7rr)dr 



where 70 i/ie f unique ) primitive element for which 7 = 7q with I G N, 
and M 7 is t/ie order of the centralizer of 7. 
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For n = 2 there are additional terms coming from each cusp and the 
formula takes the form 

y^^o(^fc,m) ~ <Wlfeo(|) = m (T^)^ [ h {r)rtanh(TTr)dr 

+ ^ 2sinh(V2) m( 7)+ ^ M^rin(ft^72) 
{7}er ch v 7/ ; { 7 }er c 7 v ; 

-^^/? (i) /io(2g J R w )exp(-2|g| J R (i) (|m| - §)) 

i=l geZ 

where is the regulator of the i 'th cusp. 
Remark 4.1. Using the identity 

e im j e j e - im 3 e 3 . 

1 - e Wj + 1 - e - ie j _ _ 2-^1 

\k\<\m 3 \ 

and taking the sum over all possible signs we get 



e ik9 3 



n-l 



E ^) = (-ir 1 II( E eifee3 )- 

^{ii}™- 1 i=i |fc|<KI 

We thus see that for n > 2 our formula coincides with the formula for 
uniform lattices given in the introduction. 

Remark 4.2. By a suitable approximation argument (see e.g., |He76[ 
Theorem 7.5]) it is enough to prove this formula for test functions 
h G PW ffi (C) which is what we will assume from now on. 

4.1. Setting up the trace formula. We now review the setup needed 
for proving the hybrid trace formula. In addition to our test function, 
ho, we fix n — 1 auxiliary functions hj G PW W (C) for j — 1, . . . , n — 1. 
We only require the auxiliary test functions to satisfy that hj(i(\rrij\ — 
|)) 7^ and that hj is supported on (—6,5) for some small fixed 5 < 
minj R^ (the latter condition is used only when n = 2 in order to 
simplify some computations). To simplify notation we also set h n = ho 
and consider the multivariable function h(r) = hi(r\) • • • h n (r n ). 

For m G Z ra let f mj , j = l,...,n denote a point-pair invariants 
of weight rrij such that S mj f Tnj = hj and consider the multivariable 
functions f m (g,g') = Ylj fm^gj, g'j)- We denote by F m the operator on 
L 2 (T\G, m) with kernel 

F m (g,g') = ^2f m (g,jg'), 
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that is 

Fmip(g) = / F m (g,g')ij;(g')dg' 

where J--p C G is a fundamental domain for Y\G. 

Let Em(g, s,q), i — 1, . . . , k denote the Eisenstein series correspond- 
ing to each of the cusps, and consider the operator H m on L 2 (T\G,m) 
given by the kernel 

(4.1) 

H m (g,g') = 

E [ h(t + n V (Hq))E^\g,± + l t, q )E%(g',±- i t,- q )dt. 

i=l q&Z"- 1 JR 

The same argument as in the proof of |Ef87t Theorem 9.7] shows 
that F m — H m is of trace class and that 



tr(F m -H m ) = / (F m (g,g) - H m (g,g))dg 
= d(X } m)h(r x ), 

AeA(m) 

where r A = (r Al , . . . , r A J with Xj = \ + r\.. 

For the hybrid trace formula we are only interested in the eigen- 
values A for which Xj = \m,j\(l — \mj\) for j = l,...,n — 1. More 
generally, for any subset J C {1, . . . ,n} we define the corresponding 
set of eigenvalues 

A(m, J) = {X G A(m)|Vj G J, Xj = \mj\(l - \rrij\)}. 

We note that the eigenfunctions corresponding to eigenvalues in A(m) \ 
A(m, J) can be obtained as lifts of eigenfunctions of lower weights. This 
was made precise in |KelOt Corollary 2.1] stating in particular 

Proposition 4.1. Form G Z ra ; let a = sgn(m) G {0, 1,— l} n and 

Jm — J a — {j\ m j 7^ 0}. Then for any function \1/ on M. n for which the 
sum on the right absolutely converges we have 

£ d{m, A)*(A) + <W(-1)I J "^(0) 

AeA(m,J m ) 

= E E d(m-av,XMX). 

^e{o,i} n \eh(m-cju) 
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In particular, when J m = {1, . . . ,n — 1} we get 
(4.2) / (F r ^ m ,(g,g)-H^ m ,(g,g))dg = 



E (- 1 )' Jl/| E d(A,m-ai/)/i(r A ) 

ve{0,l} n AeA(m-o7/) 

E d(m, X)h(r x ) + 5m,a(-l) |Jml MD 

AeA(m,J m ) 



'3 V 2 

3=1 

Note that, up to the factor YYjZi hA ^~?~ )i ^ ne r ight hand side is 
exactly the spectral side of the hybrid trace formula. In the following 
sections we compute the left hand side and show that it is given by the 
same factor times the geometric side. 

4.2. Truncated kernels. In order to compute the geometric side of 
the formula we truncate the kernels as in [Ef87l Chapter III] and then 
take a limit sending the cutoff parameter to infinity. To do this, let Yq 
be the function Y (n x a t k) = e ilH Hn measuring the distance into the 
cusp at infinity. Fix a large parameter A > and define the truncated 
Eisenstein series E^ A (g,s,q) given by 

E$(9, s, q) ~ 5i^l(T jg , q) - (f>% j \s, q^l^fag, q), 

if Y (rjg) > A and by Em(g, s, q) otherwise. We then define the trun- 
cated operator H m ^ A by replacing E$(g, s, q) by E^ A (g, s, q) in (14.11) . 
We also truncate the operator F m replacing it by the truncated kernel 



F m ,A(g,g') 



U q X 

u» 



g')dx, 



where we use a smooth cutoff function 
We then have that 



1 t > A 
t < A- 1 



/ (F m (g, g) - H m (g, g))dg = lim / (F m>A (g, g) - H mA (g, g))dg. 
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For fixed A, the integrals J ^ F m>A (g , g)dg and H m>A (g, g)dg con- 
verge; while both integrals blow up logarithmically as A — > oo, the 
logarithmic terms in fj. F mtA (g, g)dg and fj. H mtA (g, g)dg cancel out, 
and the limit of the difference can be computed explicitly in terms of 
the test function h. 



4.3. Alternating sums. We will take a slightly different approach, 
more suitable for computing the alternating sum 

(- 1 )'" 7 "' / {Fm-w{g,g)-H m _ av {g,g))dg. 
!/e{o,i}" Tt 



Instead of computing the difference fj. (F mjA (g, g) — H mjA (g, g))dg and 
taking A — > oo we will compute separately each of the alternating sums 

(4.3) Yl (-^ lM f Fm-*uA9,9)dg, 



Ju 



and 



(4.4) / H m _ a „, A (g,g))dg 



^e{o,i} n 

Ju 

We will see that, in each of these sums, the logarithmic terms cancel 
out and the limit as A — > oo can be computed explicitly. Moreover, 
in addition to the logarithmic blowup, many terms appearing in the 
general trace formula also cancel out, saving quite a bit of computa- 
tions. Most of this cancelation follows from the following combinatorial 
observation. 

Lemma 4.2. Let C : Z n — > C be any function satisfying that for some 
1 < j < n, the differences C(m) — C(m — ej) depend only on rrij. Then 
for any m G Z n with {j} C J m we have 



(-l) lM C{m - sgn(m» = 0. 



ue{o,i} n 

Ju 

C<7 m 



Proof. For simplicity assume that rrij > (the argument for rrij < 
is analogous). Let a = sgn(m) G {0,1,-1}™. We can separate the 
subsets of J m as a disjoint union of two sets 

{Ju C J m \j G J v } U {J v C J m \j £ J v }, 
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each having the same number of elements. We can correspondingly 
rewrite the alternating sum as 



^2 (-l) w C(m - <rv) = E (-l) ]Jul {C{m-au) - C(m - <ru - ej)) 

E (-l) lJ Mm 3 ) 



cynij) 



E (- 1 ) |J " 



Since we assumed that |J m | > 2, there is an even number of subsets 
{Jp Q Jm\j Ju}, half of them with even cardinality and half with 
odd cardinality hence X^j„cj m ( — 1)' J "' = 0. 

□ 

4.4. Continuous contribution. Following the same arguments as in 
[Ef87, Chapter III, Proposition 1.1] we have 

Proposition 4.3. For any fixed m G Z n as A — >• oo 



Ft 



2- 1 log(A)E^ ) E M21og(n«)) + ^tr($ m (|,0)) 



i=l 



~^E E / Ht + n^\qmpHm)}dt + o(l). 



i=i 

The first term, involving log(A), does not depend on m and hence 
cancels out when taking the alternating sum. Also, since P m {\) = 1 
we have that $ m (|,0) = $o(§,0), and hence the only dependence on 
m is in the C^'(m) terms. Since P m {\ + tt)P m (\ — it) = 1 we have 

<j>^\\ + it, g)^(i - it, -q) = ^\\ + it, q)^\\ - it, -q) 

+^(| + zW%) + t))<P^\\ + it, - it, -q). 
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The first term does not depend on m and for the second term, from 
unitarity of $(~ + it, q) and the fact that 4>^(s,q) = <j>^'(s, —q) we 
get that 

K 

^^(i + ft, g )^(i-ft,- g ) = l. 

3=1 

We thus see that for rrij > (respectively rrij < 0) 

= ^(5 + »W°(?) + 0) - 5^(1 + + 0) 



1 - 2| 








-|) 2 + (7T77«( ? )+t) 2 



depends only on rrij. 

Hence, by Lemma 14.21 when n > 2 there is no contribution from the 
continuous spectrum and for n = 2 we have 

/ (H {mfi)A (g, g) - H {m _ lfi)A (g, g))dg = 

itE/ h(t + ^(q)) —dt + o(l) 

In this case the groups are cyclic generated by some eW satisfying 
that = 1. Hence D« = J^^J)^ , = I log |e«||, and 

V^Kl) — ~ 2bF> )■ ^ e can ^ us rewrr t e the above term as 

(4-5) 

1 /" t / ir? m / 7rg 2|m| — 1 , 

Making a change of variables t H- £ + and changing the order of 
summation and integration (14.51) becomes 

( 4 6 ) W-irf/r^, M*) 



47T 



v?6 

Using Poisson summation we replace 
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and (14.61) becomes 



a 51 1 ■/« (M- 5 ) 2 + « 2 

Finally, using the assumption on the support of /ii we can use Lemma 
12.41 to replace 

H zl f <x*@QR®t) dt = - q nV(2\m\-l) h f - 2H-U 

7r (H-i) 2 +t 2 2 

We can thus conclude 

Proposition 4.4. For n > 2, as A — )■ oo ; 



and for n = 2 



£ (_1)W /" H mA (g,g)dg^0, 



I (%0),a(9. - #(m-l,O),.A(0» fiOMg = 



+ £ 2i?« £ h (2qR^)e- 2 ^ ) ^-hh 1 (z 2 -^)) + o(l) 

4.5. Compact contribution. Instead of using the truncated kernel 
Fm,A t° evaluate (14. 3p . it is convenient to use the original kernel, F m , 
but integrate over a truncated fundamental domain. Just as for the 
m = case (see |Ef87t page 86]) as A — > oo 



F mA (g,g)dg= F m (g, g)dg + o(l), 
where 

(4.7) J~a = {g G J- r |V (r i0 ) < A, j = 1, . . . , k} 
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To compute the term on the right expand F m as a sum over all lattice 
elements and collect together T-conjugacy classes to get 

/ F m (g,g)dg = / f™{g,ig)dg 

= / fm(g,ig)dg, 



{7} 

where the last sum is over all T-conjugacy classes, and 

F~t,A = U T£ r/r 7 T Ta- 

We now describe the contribution of the regular conjugacy classes, 
that is, the trivial class and the elliptic, mixed and hyperbolic conju- 
gacy classes that do not have a parabolic fixed point (the contribution 
of parabolic classes and hyperbolic-parabolic classes will be dealt it in 
the following sections). 

For the regular classes the orbital integrals converge and we have 

Ixrn^ / f m (g,jg)dg= f m {g,~fg)dg = vo\{T 1 \G 1 )I m {r,h), 



where I m (j;h) = I mj {"lj] hj) with 1^.(7^^) the orbital integral 
given in (12.51) . Using Lemma [2. II for these orbital integrals and taking 
the alternating sum over all neighboring weights we see that for the 
trivial element 



{-if A i m -„v{i;h) 



ve{o,i} n 

■ 2\mj\-l 



\m\ 



(47T) 



h (r)r tanh(7rr)<ir | jQ^jl 



For the mixed classes (and regular hyperbolic classes) the alternating 
sum vanishes unless 7 is elliptic-hyperbolic in which case 

{-if A i m -Ur,h) = 



ve{o,i} n 



2 sinh(£ 7 /2) 



n-l 

l) 



"J\ 2 
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or elliptic, in which case 

£ {-i)^i m ^{r,h) = 

ve{o,i}" 

ho(6 ltni 0) / 
sin(0 7 ,„/2) V/ _, 

We also note that for 7 elliptic-hyperbolic, T 7 is cyclic generated by 
some primitive element 70 and vol(r 7 \G 7 ) = £ l0 . For 7 elliptic, T 7 is 
a finite group of order M 7 and vol(r 7 \G 7 ) = 
To conclude, we have shown that 

Proposition 4.5. The contribution of the regular conjugacy classes to 
(14. 3 p zs given by 

H*vol(r\G) f ^ L t Ji^l 




(4vr) n 




^ M 7 sin(# 7 ,„/2) ml f) l HI ft A* 2 J 



{7}er c 

4.6. Parabolic contribution. We can collect together the parabolic 
elements corresponding to each cusp and compute their contribution 
separately. To simplify notation we will assume for now that we are 
dealing with the cusp at infinity and omit the superscripts enumerating 
the cusps. 

We recall [Ef871 Chapter III, Lemma 2.1] stating that we can take the 

elements n a = ^ with a E O/U 2 as a full set of representatives 

for the parabolic conjugacy classes corresponding to the cusp at 00. 
The contribution of these parabolic elements to tr(F mj ^) is then given 
by 



E 

aeO/U 2 



fm(g,n a g)dg 



where 

Foo,A = {n x a t k e \x G T Q , Y Q (g) < A}. 

Indeed, the compact support of f mj (g,l) implies that f m (g,n a g) = 
when g is close to any of the other cusps. We can thus use the domain 
^oo.a ignoring the cutoff at all the other cusps. Following the same 
argument as in [Ef87, pages 88-89] we get 
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Proposition 4.6. For any m G Z n ; as A — )■ oo, £/ie contribution to 
tr(F mjj 4) of the parabolic classes corresponding to the cusp at infinity is 
given by 

E / fm(g,n a g)dg = 
aeo/u 2 •'• 7 W 

n P (h) 

(2 n - 1 Rhg(A) + c o v)h(0) + 2 n Rh(0) V ? J + o(l), 

where I v m .{hj) is given in (12. 8p and zs i/ie Euler constant of O. 
Proof. Using the coordinates g = n x atk and the fact that 
f m (n x a t k,n a n x a t k) = f m (l,a- t n a a t ), 



we get 



E / fm(g,n a g)dg = 

y~] / / m (l,n Qe -t)e"*^ 

-,„,rrO J Y Si {a t )< A 

— I 

N{a) J N{x)> EM. 



a&O/U 2 JT ™,A 
V 



= V T77 — T / fm(l,n>x)dx 
^ N( rv\ N( a ) JmK ' ' 



aeO/U 2 

where N(a) = \a± ■ ■ • oi n \, N(x) = \xi . . .x n \ and the integral is over 
all x e R 7 ] with N(x) < N(a). Exchanging the order of summation 
and integration we get 

Yl / fm(g,n a g)dg = 

N(a)<AN(x) 

Using [Ef87t Proposition 2.2] stating that as A — > oo 

i o2n— 1 p 

£ T?TT = M^V(x)) + 2" Co + o(l), 

N(a)<AN(x) 

together with the identity 

2 n / f m (l,n x )dx= f m (l,n x )dx = h(0), 
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(which follows directly from (12. 2(1 ) we get that 



^2 / fm(g,n a g)dg = 

(2 n ' 1 R\og(A) + c o v)h(0) + 2 2n - l R [ f m (l,n x )log(N(x))dx. 



Finally, we can replace 

T- 1 [ / m (l, n x ) log(N(x))dx = T- 1 V / / m (l, n x ) log( Xj )dx 

n 

concluding the proof. □ 

When taking the alternating sum over all neighboring weights, the 
first term (involving log(A)) does not depend on m and hence cancels 
out. For the second term, let 

C p (m)=2 n Rh(0)J2 Ini '" Jl 



t Mo) 

then 

C p {m) - C p {m - e 3 ) = 2 n Rh{0) Wj 3 \ — - , 

hj(0) 

depends only on rrij. Hence, from Lemma l4~2l for m = (mi, . . . , m n _i, 0) 
with n > 2 we have that Y^j cj ( — l)' J,/ 'C(m — av) = 0, while for n = 2 
we have 

C p (m, 0) — C p {m — a, 0) = 

- M o ,0)^-i/ B j^^f^ - «M0)M^) 

Repeating the same argument for each cusp and collecting the cor- 
responding terms we can conclude 

Proposition 4.7. When n > 2 the parabolic classes do not contribute 
to (14. 3p and for n = 2 their contribution is given by 
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4.7. Hyperbolic-parabolic contribution. Here again, we compute 
the contribution of the hyperbolic-parabolic classes corresponding to 
each cusp separately; to simplify notation we assume we are dealing 
with the cusp at infinity and drop the superscripts. 

We recall the discussion in |Ef87, Chapter II Section 3], showing 
that any hyperbolic element fixing the cusp at oo is conjugated in 

r to some element of the form 7<?,« — I 

u q = ef • • ■ e^Zi with e%, . . . , e n _i generators for U. In particular, to 
get a full set of representatives we can take all q G Z" _1 /{±1} and for 
any such q there are finitely many values of a (see |Ef87, Chapter II 
Proposition 3.3]). In what follows we will calculate the contribution to 
(14.31) coming from the conjugacy class of 7 9jQ (and show that it goes to 
as A — )■ oo). 



1 , where as before 



Remark 4.3. In general, when doing this there could be some over 
counting due to hyperbolic elements fixing two different cusps. How- 
ever, since we will show that there is no contribution at all from these 
elements we do not have to worry about this issue. 

The other fixed point of j q>a (other than oo) is given by - Let 
t G G be an element sending — i ^=t to oo. We note that it is of the 

u U q —Uq 

/ * * \ 

form T — I u,q—Ug a 

j and that N(A) = YYj=i I A? I ^ s independent of 
the choice of r A (see A [Ef87l Page 92]). We also recall [Ef87l Chapter II, 
Proposition 3.1] stating that the centralizer r 7 is a free commutative 
group of rank n — 1, and that the set 



J q,ot 



{q'\j g >, a > e r 7g J, 



is a lattice in R™ -1 (in fact, the map sending jg'^' ^ 1 gives an iso- 
morphism of r 7 and L qta ). 

Let J z lq a denote a fundamental domain for T lq a \G. Using the coor- 
dinates at the cusp Y (g),Yi(g), . . . ,Y n _i(g) defined as in [Ef87| page 
51] for g = n x a t k e by the equation 



(tx\ 

t 2 



D 



/log(>o)\ 
2Y 1 



this fundamental domain is given by 



^ q , a ={9 = n x a t k\x G R n , Y G (0, oo), (Y u ..., Y n ) G R n /L qtCl }. 
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The contribution of the conjugacy class {7 g ,«} to the trace of F m ^A is 
then given by (see |Ef87l Page 94]) 



fm(g,i q ,ag)dg, 

Sa 



where 



Sa = {g e F lq , a \Y (g) < A, Y (rg) < A}. 



Recalling the relation f(z,w) = f(p z ,p w ) we can write this integral 
explicitly 



fm(g,7 q ,ag)dg = 

f m (z,u 2 q z + au q )dz 

dY 

f m (i, uli + x{u\ - 1) + u q )dY 1 ■ ■ ■ dY n ^—^-dx 

S A Y 



where we made the change of variables xa \— > <yUq,3 l ^£_t!^£^ resulting 
in the new fundamental domain 



S A = {x E W 1 , {Y 1 , ...,Y n )e R n /L q>a , B A {x) <Y < A}, 



with Ba(x) = zfr^ -u 2 TlAtf + 1) 1 - We can first integrate over 
Yi, . . . , Y n _i to get \L q>a \ = vol(lR n /L gja ) and then preform the integral 
over Y to get 



log(A) - hg(B A (x)) = 2 \og(A) + 2 log(iV( q q )) + ]T log(^ 2 + 1). 

3=1 
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We are thus left with the integral over ieK" giving 

fm{g,iq,ag)dg = 

= 2|L g , a |(log(A) +log(JV( 9 q ))) / f m (n X} a 2log{lUql) n x )dx 

n „ 

+\L qta \y2 / fm{n x ,a 2 \ og (\ Uq \)n x ) log(l + xf)dx 

3 1 i/ j 

Note that, for fixed A, the sum over all representatives 

fm{g,lq, a g)dg\ < oo, 



S A 



absolutely converges. We can thus take the alternating sum over the 
neighboring weights for each of the terms individually. The only de- 
pendence on the weight m is in the orbital integral I^f.(2 log(|M gj |); hj) 
(which depends only on rrij). Hence, Lemma [4.21 implies that this van- 
ishes when taking the alternating sum whenever n > 2. For n = 2 
we recall that U is generated by one element e, that R = \ log |e|| and 
that u q = e q for some q G N. We thus get that 1^(2 log(|u g;2 |); hi) = 
I^(2qR; hi) which vanishes by Lemma [231 and our assumption that hi 
is supported on (—R,R). 

Remark 4.4. We used the assumption that hi is supported on (— R, R) 
in two places, once (via Lemma 12.41) when computing the contribution 
of the continuous spectrum, and again (via Lemma I2.3j) when showing 
that there is no contribution from the hyperbolic-parabolic elements. 
Without this assumption we would have gotten a contribution from the 
hyperbolic-parabolic elements that would have had to cancel out with 
additional terms added to the continuous contribution. 



4.8. Collecting all the terms. We can now collect all the terms con- 
tributing to the alternating sums in (14. 3 p and (14.41) . For n > 2 the only 
non-vanishing contribution comes from the regular classes, so, after 
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taking the limit A — > oo we get 

/ (F m ^ v (g,g)-H m ^ u (g,g))dg 

^£{0,1}" ^ 

(4tt)" A f-^ 2smh(£ 7 /2) 

nisi ch 

+ e (n^ 

{ 7 }er c i y 7,n/ ; 7 \i=i 

Equating this to (14. 2[) and dividing by ( 11^=1 hj(i 2 ^ mj ^ 1 )j we get the 
trace formula. 

When n = 2 we also have non trivial contributions from the contin- 
uous spectrum and parabolic elements. Note that the terms involving 
the integral L dt coming from the continuous spectrum and 

the parabolic elements cancel each other and we get 

((F {mfi) (g,g) - H {m> o)(g,g)) - (F (m - afi ) (g, g) - H {m „ afi) (g,g)))dg 

ho{r)r tanh(7rr)(ir 



(2|m| - l)vol(r\G) 



(4tt) 2 



2 wah(£J2) 1 - ^ M 7 sin(0~ 2 /2) 1 - e^- 1 

{7}er eh v 7/ y { 7 }er c 7 v 7i// y 



i=l i=l q£N 



Equating to (14.21) and dividing by hi(i 2 ^ 1 ) we get the trace formula 
for n = 2. 

4.9. Another proof of Lemma 12.21 We now show how the ideas 
from this section, when applied to the modular group T = PSL 2 (Z), 
give another cute proof of the identity 

2m- 1 f h(t) ^ 1 j,. 2m _i> 



8tt J R (m-^) 2 + t 2 4 

for m G N, where IfJJi) is given in (12.81) . 

To do this, fix a test function h G PW W (C), let f m denote a weight 
m point pair invariant with S m f m = h, and let F m denote the corre- 
sponding integral operator on L 2 (r\PSL 2 (R), m). Let H m be defined 
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as in ( 14. ip using the Eisenstein series at the single cusp at infinity. We 
now compute the difference 

tr(.F m — H m ) — tr(F TO _i — H m _i), 

in two different ways. 

On one hand, applying the lowering operator C~ we see that all eigen- 
functions contributing to tr(F m — H m ) with eigenvalues A 7^ m(l — m) 
will cancel with corresponding eigenfunctions contribution to tr(F m _! — 
iJ m _i); when computing tr(F — H ) we also get a contribution from 
the constant eigenfunction which does not cancel out with any of the 
eigenfunctions contributing to tr(Fi — Hi). We thus get that 

tr(F m - H m ) - tr(F m _! - H m ^) = h{i^) (dim M m (T) - 5 m>1 ), 

where 

M m {T) = {ijE L 2 (r\H,m)|(A m + m(l - m))^ = 0}. 

On the other hand, following the same arguments as above, noting 
that in this case there are no hyperbolic-parabolic classes nor mixed 
classes we get that 

tr(F m - H m) - tr(F m _, - ff m _0 = Izgi j£ (m _*g, +f , J 

V v 7 4tt ^ My 1 - / v 2 ; 

v {7}er c 7 7 

+2{F m {h)-P m _ 1 {h)), 

where the first term comes from the continuous spectrum, the second 
term comes from the trivial class and the elliptic class, and the last 
term comes from the parabolic elements. 
Equating the two we get the identity 



2m - 1 f h(t) h(i 



2m- 1 - 



'm - - — 1 (m _ V2)2 + ¥ * + 

(O m _ 1 . _ 1 p imd 1 \ 

dta M m (D - ^ - vol^) — - £ M.^) 
{7}er c 7 7 

Now, for the modular group we have that vol(J-r) — f an d there are 
3 elliptic conjugacy classes (one of order 2 and two of order 3). Hence 

vol^ r )^i + V J_^]__ /[f]-f m=l (mod6) 
° 4tt + ^ M 7 1 - e*7 ~ 1 f + I m ^ 1 (mod 6) ' 

{ 7 }er c 7 k LbJ 2 y 
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Moreover, we recall that M m (T) is isomorphic to the space of cusp 
forms of weight 2m which, for the modular group, is of dimension 

{0 m = 1 

[f]-l m>l,m = l (mod 6) 
[f] m>l,m^l (mod 6) 

Combining the two together we get that 

dim(M m (r)) - cW - vol(Jr)^^ - £ jf^—^ = 

{7} 7 



and plugging this back in gives 

2m — 1 / 

concluding the proof. 



tt J R (m - 1/2)2 + 1 2 4 V 2 h 



5. Counting and Equidistribution 

The results on counting elliptic-hyperbolic classes and on the dis- 
tribution of their elliptic parts follow from Theorem [3] exactly as in 
[KelO] . Specifically, we note that the family of functions 

{H am {e)\meW-\ ae{±l} n - 1 }, 

form an orthogonal basis of L 2 ((IR/27rZ) n_1 , /i n_1 ) and we can decom- 
pose any function in this space as 

/(0) = ^a / (<7m)ff om (0), 

m,cr 

with 

,n-li 



o/H = J f{e)H am {e)d^--\e). 

Using the same argument as in the proof of [KelOt Proposition 3.3], and 
noting that for congruence groups we have the bound Xo(m) > \ — (^) 2 
for the spectral gap, we get 

Proposition 5.1. For any smooth f e C co ((R/2nZ) n - 1 ) 
I E o l -°ln"L - 2neX/ V(/)l « J\\f\LC(f)e^ + C(f)e*'« 

where C(f) = Ea,m |m|*|o/(<7m)|. 
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Remark 5.1. For n > 2 the hybrid trace formula in the nonuniform 
case is identical to the uniform case and the proof goes over with out 
any changes. When n = 2 there are additional terms in the nonuniform 
case, however, it is easy to see that these terms are negligible for these 
asymptotic estimates (the new terms are essentially equivalent to the 
contribution of a finite number of short primitive geodesies). 

Theorem [2] now follows from Proposition 15.11 by a standard summa- 
tion by parts argument. Corollary [1] can be obtained from Proposition 
15.11 by using the Beurling-Selberg functions to approximate the indica- 
tor function of an interval (see the proof of [KelOl Corollary 3.1] for 
more details). We conclude this section by recalling the relation with 
quadratic forms in more detail. 

Let F denote a totaly real number field of degree [F : Q] = n > 2 
with ring of integers Op. We fix an embedding i : F ■=->■ R n and 
correspondingly identify PSL 2 (0_f) as a lattice in PSL 2 (M)™. Given a 
binary quadratic form q(x, y) = ax 2 + bxy + cy 2 with a, b, c G Op, its 
discriminant is D = b 2 — 4ac, its divisor is the ideal (a, b, c), its splitting 
field is F{y/~D) and its primitive discriminant is the ideal d = , ^ \ 2 . 
The splitting field and the primitive discriminant are invariant under 
the equivalence relation q ~ tq o 7 for t G F* and 7 6 Tp, and we 
denote by h(D, d) the number of equivalence classes with splitting field 
F(y/~D) and primitive discriminant d. 

For any non-square D G Of-, and primitive discriminant d C Of let 
On,d denote the quadratic order in F{yD) with relative discriminant 
d, that is, 

D , d = l^±^eO Fm :d\(u>D) 

Let 0\> d denote the group of relative norm one elements in Oo,d'-, ex- 
plicitly this is all e = its^R e Q D d w ith t 2 - AD = 4. This group is 
naturally isomorphic to the group of automorphs of a quadratic form 
q = ax 2 + bxy + cy 2 with discriminant D and primitive discriminant d. 

/ t-bu — cu \ 

Specifically, for any e = t+u ^ D g O x D d , we have that 7 = ( £ t+hu \ 

satisfies q = q o 7, and moreover any automorph of q is of this form 
(see |Ef87l Theorem 5.6]). The map sending the equivalence class of q, 
to the conjugacy class of the centralizer {T 7 }, where 7 G PSL 2 (Cf) is 
any automorph of q, is a correspondence between equivalence classes of 
forms, and centralizers of mixed elements (see [Ef87| Propositoin 7.1]). 
Moreover, if Lj(D) > in exactly k places, then ©^/{il} — Z fc and 



HYBRID TRACE FORMULA 



35 



it corresponds to the centralizer of an element that is hyperbolic in k 
places and elliptic in the rest (see [Ef87j Theorem 5.7]). 

In particular, for D G Op with i n (D) > and ij(D) < for 
j < n and ideal d C Op,^ let ep>,d G O x D d denote the unique gen- 
erator of O l D J '{±1} satisfying that i. n (e Dd ) > 1 (the other generators 
are then ±e^ d ). Since the centralizer of an elliptic- hyperbolic ele- 
ment is generated by two primitive elements (7 and 7" 1 ), there is a 
correspondence between equivalence classes of quadratic forms with 
splitting field F(\/T)) and primitive discriminant d, and pairs of primi- 
tive elliptic-hyperbolic conjugacy classes {7}, {7 -1 } with = i n (^D,d) 
and e 7j = l^d,^ 1 for j < n. Note that the choice of each sign 
tj^D^ 1 in this correspondence in ambiguous, however, for a test 
function / G (S 1 ) n ^ 1 that is invariant under 6j 1— > ej 1 for all j this 
ambiguity is irrelevant and Corollary [2] follows from Theorem |2] ap- 
plied to T = PSL 2 (C F ). 
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